Systems and methods for multi-pressure interaction on touch-sensitive surfaces

ABSTRACT

Systems and methods for multi-pressure interaction on touch-sensitive surfaces are disclosed. One disclosed embodiment of a method comprises receiving a first sensor signal from a touch-sensitive input device in response to a first contact of a first object on the touch-sensitive input device, the first sensor signal comprising a first location and a first pressure of the first contact, receiving a second sensor signal from the touch-sensitive input device in response to a second contact of a second object on the touch-sensitive input device substantially simultaneously with the first contact, the second sensor signal comprising a second location of the second contact and a second pressure of the second contact, generating a signal based at least in part on the first sensor signal and the second sensor signal, the signal configured to cause a haptic effect, and outputting the signal.

FIELD OF THE INVENTION

The present disclosure relates generally to systems and methods formulti-pressure interaction on touch-sensitive surfaces.

BACKGROUND

With the increase in popularity of handheld devices, especially mobilephones having touch-sensitive surfaces (e.g., touch screens), physicaltactile sensations which have traditionally been provided by mechanicalbuttons no longer apply in the realm of these new generations ofdevices. Instead, haptic effects may be output by handheld devices toalert the user to various events. Such haptic effects may includevibrations to indicate a button press, an incoming call, or a textmessage, or to indicate error conditions.

SUMMARY

Embodiments of the present invention provide systems and methods formulti-pressure interaction on touch-sensitive surfaces. For example, inone embodiment of a method disclosed herein, the method comprisesreceiving a first sensor signal from a touch-sensitive input device inresponse to a first contact of a first object on the touch-sensitiveinput device, the first sensor signal comprising a first location and afirst pressure of the first contact, receiving a second sensor signalfrom the touch-sensitive input device in response to a second contact ofa second object on the touch-sensitive input device substantiallysimultaneously with the first contact, the second sensor signalcomprising a second location of the second contact and a second pressureof the second contact, generating a signal based at least in part on thefirst sensor signal and the second sensor signal, the signal configuredto cause a haptic effect, and outputting the signal. In anotherembodiment, a computer-readable medium comprises program code forcausing a processor to execute such a method.

These illustrative embodiments are mentioned not to limit or define theinvention, but rather to provide examples to aid understanding thereof.Illustrative embodiments are discussed in the Detailed Description,which provides further description of the invention. Advantages offeredby various embodiments of this invention may be further understood byexamining this specification.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated into and constitute apart of this specification, illustrate one or more examples ofembodiments and, together with the description of example embodiments,serve to explain the principles and implementations of the embodiments.

FIG. 1 shows a multi-pressure touch-sensitive input device according toan embodiment of the present invention;

FIG. 2 illustrates a multi-pressure touch-sensitive input deviceaccording to an embodiment of the present invention;

FIG. 3 illustrates a flow chart directed to a method of detecting andresponding to a contact on a multi-pressure touch-sensitive input devicein accordance with an embodiment of the present invention;

FIG. 4 illustrates an operation of a multi-pressure touch-sensitiveinput device in accordance with an embodiment of the present invention;

FIG. 5 illustrates a flow chart directed to a method of detecting andresponding to a contact on an multi-pressure touch-sensitive inputdevice in accordance with an embodiment of the present invention;

FIGS. 6A-6C illustrate the operation of a multi-pressure touch-sensitiveinput device in accordance with an embodiment of the present invention;

FIG. 7 illustrates an operation of a multi-pressure touch-sensitiveinput device in accordance with an embodiment of the present invention;

FIG. 8 illustrates a flow chart directed to a method of detecting andresponding to a contact on a multi-pressure touch-sensitive input devicein accordance with an embodiment of the present invention; and

FIG. 9 illustrates an operation of a multi-pressure touch-sensitiveinput device in accordance with an embodiment of the present invention.

DETAILED DESCRIPTION

Example embodiments are described herein in the context of systems andmethods for multi-pressure interaction on touch-sensitive surfaces.Those of ordinary skill in the art will realize that the followingdescription is illustrative only and is not intended to be in any waylimiting. Other embodiments will readily suggest themselves to suchskilled persons having the benefit of this disclosure. Reference willnow be made in detail to implementations of example embodiments asillustrated in the accompanying drawings. The same reference indicatorswill be used throughout the drawings and the following description torefer to the same or like items.

In the interest of clarity, not all of the routine features of theimplementations described herein are shown and described. It will, ofcourse, be appreciated that in the development of any such actualimplementation, numerous implementation-specific decisions must be madein order to achieve the developer's specific goals, such as compliancewith application- and business-related constraints, and that thesespecific goals will vary from one implementation to another and from onedeveloper to another.

Illustrative Operation of a Multi-Pressure Touch-Sensitive Input Device

Referring to FIG. 1, FIG. 1 shows a multi-pressure touch-sensitive inputdevice 100 according to an embodiment of the present invention. Thedevice 100 displays a portion of a web page to a user. In thisillustrative embodiment, a user may navigate the page using multi-touch,multi-pressure inputs on the touch-sensitive surface 120. For example,if the user touches the touch-sensitive surface 120 substantiallysimultaneously with two fingers and applies more pressure with a fingerlocated nearer the bottom of the screen than the other finger, thedevice 100 will cause the display to scroll down the web page.Alternatively, the user could apply more pressure with a finger locatednearer the top of the screen than the other finger to cause the device100 to scroll up the web page. In some embodiments, the rate or speed ofscrolling is based at least in part on two or more pressures. Forexample, the rate of scrolling may be a function of the difference intwo pressures. In one embodiment, a haptic effect is output with afrequency or magnitude corresponding to the rate of scrolling.

In addition to scrolling the web page based on the user-appliedmulti-pressure inputs, the device 100 also outputs haptic effects toindicate the action taken in response to the input. For example, whilescrolling down the web page, the device 100 may output a haptic effectthat seems to travel from the top of the device 100 to the bottom of thedevice 100, and cycle repeatedly while the user continues to scroll theweb page. Or if the user is scrolling up the web page, the haptic effectstarts at the bottom of the device 100 and seems to travel toward thetop of the device 100, and cycle repeatedly while the user continues toscroll the web page. Thus, a user is able to provide multi-touch,multi-pressure input to interact with a device 100 and receive hapticfeedback based on the input.

This illustrative example is given to introduce the reader to thegeneral subject matter discussed herein. The invention is not limited tothis example. The following sections describe various additionalnon-limiting embodiments and examples of devices, systems, and methodsfor multi-pressure interaction on touch-sensitive surfaces.

Illustrative Multi-Pressure Touch-Sensitive Input Device

Referring now to FIG. 2, FIG. 2 illustrates a multi-pressuretouch-sensitive input device 200 according to an embodiment of thepresent invention. In the embodiment shown in FIG. 2, the device 200comprises a housing 205, a processor 210, a memory 220, atouch-sensitive display 230, an actuator 240, and a network interface250. The processor 210 and the memory 220 are disposed within thehousing 205. The touch-sensitive display 230, which comprises or is incommunication with a touch-sensitive surface, is partially disposedwithin the housing 205 such that at least a portion of thetouch-sensitive display 230 is exposed to a user of the device 200. Insome embodiments, the touch-sensitive display 230 may not be disposedwithin the housing 205. For example, the device 200 may be connected toor otherwise in communication with a touch-sensitive display 230disposed within a separate housing.

In the embodiment shown in FIG. 2, the touch-sensitive display 230 is incommunication with the processor 210 and is configured to providesignals to the processor 210, which is also in communication with memory220. The memory 220 stores program code or other data, or both, for useby the processor 210 and the processor 210 executes program code storedin memory 220 and receives signals from the touch-sensitive display 230.The processor 210 is also configured to output signals to cause thetouch-sensitive display 230 to output images. In the embodiment shown inFIG. 2, the processor 210 is in communication with the network interface250 and is configured to receive signals from the network interface 250and to output signals to the network interface 250 to communicate withother components or devices. In addition, the processor 210 is incommunication with actuator 240 and actuator 260 and is furtherconfigured to output signals to cause actuator 240 or actuator 260, orboth, to output one or more haptic effects. Furthermore, the processor210 is in communication with speaker 270 and is configured to outputsignals to cause speaker 270 to output sounds. In various embodiments,the device 200 may comprise or be in communication with fewer oradditional components or devices. A detailed description of thecomponents of the device 200 shown in FIG. 2 and components that may bein association with the device 200 is described below.

The multi-pressure touch-sensitive input device 200 can be any devicethat comprises or is in communication with a touch-sensitive surfacethat is capable of detecting pressures associated with at least twocontacts on the touch-sensitive surface. For example, the device 200 ofFIG. 2 includes a touch-sensitive display 230 that comprises atouch-sensitive surface. In some embodiments, a touch-sensitive surfacemay be overlaid on the display 230. In other embodiments, the device 200may comprise or be in communication with a display and a separatetouch-sensitive surface.

In some embodiments, one or more touch-sensitive surfaces may be on oneor more sides of the device 200. For example, in one embodiment, atouch-sensitive surface is disposed within or comprises a rear surfaceof the device 200. In another embodiment, a first touch-sensitivesurface is disposed within or comprises a rear surface of the device 200and a second touch-sensitive surface is disposed within or comprises aside surface of the device 200. Furthermore, in embodiments where thedevice 200 comprises at least one touch-sensitive surface on one or moresides of the device 200 or in embodiments where the device 200 is incommunication with an external touch-sensitive surface, the display 230may or may not comprise a touch-sensitive surface. In some embodiments,one or more touch-sensitive surfaces may have a flexible touch-sensitivesurface. In other embodiments, one or more touch-sensitive surfaces maybe rigid. In various embodiments, the device 200 may comprise bothflexible and rigid touch-sensitive surfaces.

In various embodiments, the device 200 may comprise or be incommunication with fewer or additional components than the embodimentshown in FIG. 2. For example, in one embodiment, the device 200 is notin communication with speaker 270 and does not comprise actuator 240. Inanother embodiment, the device 200 does not comprise a touch-sensitivedisplay 230 or a network interface 250, but comprises a touch-sensitivesurface and is in communication with an external display. In otherembodiments, the device 200 may not comprise or be in communication withan actuator at all. Thus, in various embodiments, the multi-pressuretouch-sensitive input device 200 may comprise or be in communicationwith any number of components, such as in the various embodimentsdisclosed herein as well as variations that would be apparent to one ofskill in the art.

The housing 205 of the device 200 shown in FIG. 2 provides protectionfor at least some of the components device 200. For example, the housing205 may be a plastic casing that protects the processor 210 and memory220 from foreign articles such as rain. In some embodiments, the housing205 protects the components in the housing 205 from damage if the device200 is dropped by a user. The housing 205 can be made of any suitablematerial including but not limited to plastics, rubbers, or metals.Various embodiments may comprise different types of housings or aplurality of housings. For example, in some embodiments, themulti-pressure touch-sensitive input device 200 may be a cell phone,personal digital assistant (PDA), laptop, tablet computer, desktopcomputer, digital music player, gaming console, gamepad, medicalinstrument, etc. In other embodiments, the device 200 may be embedded inanother device such as, for example, the console of a car.

In the embodiment shown in FIG. 2, the touch-sensitive display 230provides a mechanism for a user to interact with the multi-pressuretouch-sensitive input device 200. For example, the touch-sensitivedisplay 230 detects the location and pressure of a user's finger inresponse to a user hovering over, touching, or pressing thetouch-sensitive display 230 (all of which may be referred to as acontact in this disclosure). In some embodiments, the touch-sensitivedisplay 230 may comprise, be connected with, or otherwise be incommunication with one or more sensors that determine the location,pressure, or both, of one or more contacts on the touch-sensitivedisplay 230. For example, in one embodiment, the touch-sensitive display230 comprises or is in communication with a mutual capacitance system.In another embodiment, the touch-sensitive display 230 comprises or isin communication with an absolute capacitance system. In someembodiments, the touch-sensitive display 230 may comprise or be incommunication with a resistive panel, a capacitive panel, infrared LEDs,photodetectors, image sensors, optical cameras, or a combinationthereof. Thus, the touch-sensitive display 230 may incorporate anysuitable technology to determine a contact on the touch-sensitivesurface 120 such as, for example, resistive, capacitive, infrared,optical, thermal, dispersive signal, or acoustic pulse technologies, ora combination thereof.

In the embodiment shown in FIG. 2, actuators 240 and 260 are incommunication with the processor 210 and are configured to provide oneor more haptic effects. For example, in one embodiment, when anactuation signal is provided to actuator 240, actuator 260, or both, bythe processor 210, the respective actuator 240, 260 outputs a hapticeffect based on the actuation signal. For example, in the embodimentshown, the processor 210 is configured to transmit an actuator signal toactuator 240 comprising an analog drive signal. However, the processor210 is configured to transmit a command to actuator 260, wherein thecommand includes parameters to be used to generate an appropriate drivesignal to cause the actuator 260 to output the haptic effect. In otherembodiments, different signals and different signal types may be sent toeach of one or more actuators. For example, in some embodiments, aprocessor may transmit low-level drive signals to drive an actuator tooutput a haptic effect. Such a drive signal may be amplified by anamplifier or may be converted from a digital to an analog signal, orfrom an analog to a digital signal using suitable processors orcircuitry to accommodate the particular actuator being driven.

An actuator, such as actuators 240 or 260, can be any component orcollection of components that is capable of outputting one or morehaptic effects. For example, an actuator can be one of various typesincluding, but not limited to, an eccentric rotational mass (ERM)actuator, a linear resonant actuator (LRA), a piezoelectric actuator, avoice coil actuator, an electro-active polymer (EAP) actuator, a memoryshape alloy, a pager, a DC motor, an AC motor, a moving magnet actuator,an E-core actuator, a smartgel, an electrostatic actuator, anelectrotactile actuator, or any other actuator or collection ofcomponents that perform the functions of an actuator. Multiple actuatorsor different-sized actuators may be used to provide a range ofvibrational frequencies, which may be actuated individually orsimultaneously. Various embodiments may include a single or multipleactuators and may have the same type or a combination of different typesof actuators.

In various embodiments, one or more haptic effects may be produced inany number of ways or in a combination of ways. For example, in oneembodiment, one or more vibrations may be used to produce a hapticeffect, such as by rotating an eccentric mass or by linearly oscillatinga mass. In some such embodiments, the haptic effect may be configured toimpart a vibration to the entire device or to only one surface or alimited part of the device. In another embodiment, friction between twoor more components or friction between at least one component and atleast one contact may be used to produce a haptic effect, such as byapplying a brake to a moving component, such as to provide resistance tomovement of a component or to provide a torque. In other embodiments,deformation of one or more components can be used to produce a hapticeffect. For example, one or more haptic effects may be output to changethe shape of a surface or a coefficient of friction of a surface. In anembodiment, one or more haptic effects are produced by creatingelectrostatic forces that are used to change friction on a surface. Inother embodiments, an array of transparent deforming elements may beused to produce a haptic effect, such as one or more areas comprising asmartgel.

In FIG. 2, the network interface 250 is in communication with theprocessor 210 and provides wired or wireless communications, from thedevice 200 to other components or other devices. For example, thenetwork interface 250 may provide wireless communications between thedevice 200 and a wireless speaker or a wireless actuation device. Insome embodiments, the network interface 250 may provide communicationsto one or more other devices, such as another device 200, to allow usersto interact with each other at their respective devices. The networkinterface 250 can be any component or collection of components thatenables the multi-pressure touch-sensitive input device 200 tocommunicate with another component or device. For example, the networkinterface 250 may comprise a PCI network adapter, a USB network adapter,or an Ethernet adapter. The network interface 250 may communicate usingwireless Ethernet, including 802.11 a, g, b, or n standards. In oneembodiment, the network interface 250 can communicate using Bluetooth,CDMA, GSM, TDMA, FDMA, or other wireless technology. In otherembodiments, the network interface 250 may communicate through a wiredconnection and may be in communication with one or more networks, suchas Ethernet, token ring, USB, FireWire 1394, etc. And while theembodiment shown in FIG. 2 comprises a network interface 250, otherembodiments may not comprise a network interface 250.

Illustrative Method of Detecting and Responding to a Contact

Referring now to FIG. 3, FIG. 3 illustrates a flow chart directed to amethod 300 of detecting and responding to a contact on a multi-pressuretouch-sensitive input device 100 in accordance with an embodiment of thepresent invention. The method shown in FIG. 3 will be described withrespect to the device shown in FIG. 2.

The method 300 begins in block 310 when a sensor signal is received. Forexample, in one embodiment, the processor 210 receives a signal from thetouch-sensitive display 230 when a user contacts the touch-sensitivedisplay 230 and the signal includes information associated with an inputon—or a status of—the touch-sensitive display 230 such as the x, ylocation and pressure of a contact on the touch-sensitive display 230.In other embodiments, the processor 210 receives a plurality of sensorsignals. For example, the processor 210 may receive a first signalincluding information associated with a first input on thetouch-sensitive display 230, a second signal including informationassociated with a second input on the touch-sensitive display 230, and athird signal including information associated with a third input on thetouch-sensitive display 230. In one embodiment, the processor 210receives a first signal including information containing the x, ylocation of a contact on the touch-sensitive display 230 and a secondsignal including information containing the pressure of the contact. Inanother embodiment, the processor 210 receives a first signal includinginformation containing the x, y locations of two contacts on thetouch-sensitive display 230 and a second signal includes informationcontaining pressures of the two contacts. The processor 210 may receivea single signal that includes information associated with two or moreinputs on the touch-sensitive display 230. For example, in oneembodiment, the processor 210 receives a single signal that includes thex, y location and pressure of a first contact and the x, y location andpressure of a second contact.

As discussed above, in one embodiment, the processor 210 receives asignal from the touch-sensitive display 230. In some embodiments, thedevice 200 may comprise a touch-sensitive surface separate from, orinstead of, a touch sensitive display 230. In such an embodiment, theprocessor 210 may receive sensor signals(s) from the touch-sensitivesurface, or if a plurality of touch-sensitive surfaces are employed,from one or more of the plurality of touch sensitive surfaces.

In some embodiments, the processor 210 may receive one or more sensorsignals from the network interface 250. For example, in one embodiment,the network interface 250 is in communication with and receivesinformation from one or more components or devices, or both. In thisembodiment, the network interface 250 sends one or more signals to theprocessor 210 that contain information from the other components ordevices, or both. For example, the network interface 250 may receive asignal from another multi-pressure touch-sensitive input device and thesignal may contain information regarding an input on a touch-sensitivedisplay of the other device. The network interface 250 may sendinformation regarding the input on the display of the other device tothe processor 210. In another embodiment, the network interface 250receives a signal from a wireless touch-sensitive surface that is incommunication with the network interface 250 and the network interface250 sends one or more signals containing information about an input onor the status of the touch-sensitive surface to the processor 210.

In other embodiments, the network interface 250 may receive a pluralityof sensor signals from one or more components or devices incommunication with the network interface 250 and can send one or moresignals to the processor 210. For example, in one embodiment, thenetwork interface 250 is in communication with a wirelesstouch-sensitive surface and another multi-pressure touch-sensitive inputdevice. In such an embodiment, the network interface 250 may receive onesignal from the wireless touch-sensitive surface and another signal fromthe multi-pressure touch-sensitive input device. In addition, thenetwork interface 250 may send one or more signals containinginformation from the wireless touch-sensitive surface or from the othermulti-pressure touch-sensitive input device, or both, to the processor210. Thus, the processor 210 may receive one or more signals from boththe touch-sensitive display 230 and the network interface 250. Forexample, in one embodiment, the processor 210 receives a first signalfrom the touch-sensitive display 230 containing information about aninput on the touch-sensitive display 230 and the processor 210 receivesa second signal from the network interface 250 containing informationabout an input on the display of another multi-pressure touch-sensitiveinput device that is in communication with the network interface 250.

As discussed above, in one embodiment, the processor 210 receives asignal when a user contacts the touch-sensitive display 230. In such anembodiment, the processor 210 may receive a signal from thetouch-sensitive display 230 only when an input is made on the display.Or the processor 210 may receive a signal from the touch-sensitivedisplay 230 when an input is initially made on the touch-sensitivedisplay 230 and when a change to an existing input is made. For example,the processor 210 may receive one or more signals when a user contactsthe touch-sensitive display 230 and each time the user moves the contactalong the touch-sensitive display 230. In other embodiments, theprocessor 210 may receive successive signals from the touch-sensitivedisplay 230 for the entire duration of one or more contacts. In oneembodiment, the processor 210 receives a signal from the touch-sensitivedisplay 230 at specified time intervals. For example, the processor 210may receive a signal from the touch-sensitive display 230 periodically,such as every 0.1 ms. In other embodiments, the processor 210 receives asignal containing status information from the touch-sensitive display230 regardless of whether a contact is made on the touch-sensitivedisplay 230. For example, in one embodiment, the processor 210 receivessuccessive signals from the touch-sensitive display 230 at a specifiedtime intervals regardless of whether a contact is made on thetouch-sensitive display 230, but if a contact exists on thetouch-sensitive display 230 the signal may contain information regardingthe contact such as the location and pressure of the contact.

In the embodiment discussed above, the signal that the processor 210receives includes information associated with an input on—or a statusof—the touch-sensitive display 230 such as the x, y location andpressure of a contact on the touch-sensitive display 230. In variousembodiments, a signal that is received by the processor 210 can provideinformation relating to one or more contacts on the device 200,information relating to a component of the device 200, or informationrelated to other components or devices that the processor 210 can use todetermine a contact. For example, in one embodiment a signal containsinformation indicating that a contact has occurred. In anotherembodiment, the signal may contain the change in pressure of a contactfrom a previous measurement to the current measurement. Similarly, asignal may contain information regarding the change in location of acontact from a previous location. In various embodiments, a signal cancontain data including, but not limited to, location data, contact data,interaction data, gesture data, duration data, pressure data, thermaldata, waveform data, capacitive data, infrared data, photodetectiondata, optical data, other data necessary or relevant in determining acontact.

Referring again to method 300, once a sensor signal has been receivedthe method 300 proceeds to block 320. In block 320 a contact isdetermined. As discussed above, in one embodiment, the processor 210only receives a signal once a contact is made with the touch-sensitivedisplay 230. Thus, in this embodiment, the display 230 receives a sensorsignal, determines a contact, and sends a signal to the processor 210.The processor 210, on the other hand, does not have to determine acontact because the processor 210 only receives a signal from thedisplay 230 once a contact has been determined. Thus, in someembodiments, the display 230 receives sensor signals as specified inblock 310 and determines a contact as specified in block 320 and theprocessor determines a response as specified in block 330.

In some embodiments, the processor 210 determines whether a contact hasoccurred as specified in block 320. For example, a display may receivesensor signals as specified in block 310 and the display may sendinformation associated with the sensor signals to the processor 210,either directly if the display is in communication with the processor210 or through the network interface 250, which the processor 210receives and uses to determine whether a contact has occurred asspecified in block 320. In one embodiment, the information that theprocessor 210 receives comprises an instruction specifying that acontact has occurred. In another embodiment, the information that theprocessor 210 receives is indicative of whether a contact has occurred.For example, if the processor 210 receives information containing an xcoordinate, a y coordinate, and a pressure, the processor 210 may beable to use this information to determine that a contact has occurred.In another embodiment, the processor 210 receives pressure informationat periodic intervals that the processor 210 uses to determine whether acontact has occurred based upon changes in the pressure information. Inother embodiments, if the pressure information the processor 210receives is less than a threshold pressure the processor 210 maydetermine that a contact has not occurred and if the pressure is greaterthan or equal to the threshold pressure the processor 210 may determinethat a contact has occurred.

As discussed previously, a contact with the device 200 can be made innumerous ways. For example, a contact can be made with thetouch-sensitive display 230 by one or more objects such as, for example,a single finger, multiple fingers, or a pencil. In one embodiment, acontact may include physically contacting the touch-sensitive display230 and, in another embodiment, a contact may include hovering an objectover the touch-sensitive display 230 without physically contacting thetouch-sensitive display 230. Thus, in some embodiments the processor 210can determine a contact based on a physical contact with thetouch-sensitive display 230 and, in other embodiments, the processor 210may determine a contact based on a near-contact with or object hoveringover the touch-sensitive display 230.

The device 200 may use various technologies to determine whether acontact has occurred or to obtain information related to a contact. Forexample, temperatures on or near the touch-sensitive display 230 may bemeasured to determine whether a contact has occurred. Thus, a fingerapproaching the touch-sensitive display 230 may be detected and acontact determined based at least in part on the difference in theambient temperature surrounding the device 200 and the temperature ofthe approaching finger. In one embodiment, the device 200 comprises oneor more capacitive sensors that are used to detect a contact based on anobject approaching the touch-sensitive display 230. The device 200 maycomprise other components including, but not limited to, an infraredLED, a photodetector, an image sensor, an optical camera, or acombination thereof that may be used to determine, at least in part,whether a contact on the touch-sensitive display 230 has occurred or toobtain information related to a contact. Thus, the device 200 may useany suitable technology that allows the touch-sensitive display 230 todetermine, or assists the processor 210 in determining, a contact on thetouch-sensitive display 230.

In some embodiments, the device may receive information from the networkinterface 250 which the processor 210 uses to determine whether acontact has occurred as shown in block 320. For example, the processor210 may receive information from the network interface 250 that is incommunication with another device. In one embodiment, the other devicemay send the network interface 250 information when a display associatedwith the other device receives an input and the processor 210 mayreceive information from the network interface 250 related to the inputon the other device. In some embodiments, the processor 210 may receiveperiodic information from the network interface 250 about another devicethat is in communication with the network interface. In one embodimentwhere the network interface 250 is in communication with a remotetouch-sensitive surface, the network interface 250 receives informationfrom the touch-sensitive surface and sends information to the processor210 which the processor 210 uses to determine a contact. In stillfurther embodiments, another component, such as a separatemicroprocessor or co-processor may be responsible for determining acontact and providing such information to the processor 210. In variousembodiments, software stored on the memory 220 and executed by theprocessor 210 may also be used in determining whether a contact hasoccurred, such as by implementing various techniques discussed above.

Referring again to method 300, once a contact has been determined 320,the method 300 proceeds to block 330. In block 330 a response isdetermined. As discussed above, in one embodiment, the processor 210receives a signal from the touch-sensitive display 230 when a usercontacts the touch-sensitive display 230 and the signal includes the x,y location and pressure of the contact on the touch-sensitive display230. In this embodiment, if the user is viewing a web page displayed onthe touch-sensitive display 230 of the device 200 and if the processor210 determines that the user is touching the touch-sensitive display 230substantially simultaneously with two fingers and two contacts aredetermined, and it is determined that the user is applying more pressurewith a finger located nearer the bottom of the screen than the otherfinger, the processor 210 determines that the response should be toupdate the touch-sensitive display 230 to scroll down the web page andto output a haptic effect that indicates that the page is scrolling downthe web page. Alternatively, in this embodiment, the processor 210 maydetermine that the response should be to update the touch-sensitivedisplay 230 to scroll up the web page and to output a haptic effect thatindicates that the page is scrolling up the web page, such as if theprocessor 210 detects two substantially-simultaneous contacts on thetouch-sensitive display 230 and the pressure of the contact locatednearer the top of the screen is larger than the pressure of the contactlocated nearer the bottom of the screen.

In some embodiments, the rate or speed of scrolling is based at least inpart on the pressures. For example, the scrolling rate may increase asthe difference in pressures between two contacts increases. In oneembodiment, one or more haptic effects are output corresponding to therate of scrolling, such as by vibrating a device at frequencies ormagnitudes that vary based on the rate of scrolling. Thus, in someembodiments, the processor 210 determines a response, if any, asspecified in block 330. In other embodiments, the touch-sensitivedisplay 230 determines a response, if any. In still further embodiments,another component, such as a separate microprocessor or co-processor incommunication with the processor 210, the touch-sensitive display 230,or the network interface 250 may be responsible for determining aresponse and providing such information to the processor 210 or thenetwork interface 250. In various embodiments, software stored on thememory 220 and executed by the processor 210 may also be used indetermining whether a contact has occurred.

The processor 210, touch-sensitive display 230, or other component mayuse any or all of the information received to determine a contact indetermining a response. Thus, in embodiments, components of the device200 or components in communication with the device 200 or components ofanother device in communication with the device 200 may use various dataincluding, but not limited to, location data, contact data, interactiondata, gesture data, duration data, pressure data, thermal data, waveformdata, capacitive data, infrared data, photodetection data, optical data,other data necessary or relevant in determining a response. For example,in one embodiment, pressure data for two contacts is used by theprocessor 210 to determine a response. In another embodiment, thetouch-sensitive display 230 may compare the pressure of a contactagainst a threshold pressure to determine a response. In otherembodiments, information regarding one or more contacts is sent by thedevice 200 through the network interface 250 to another device thatdetermines a response, if any, and sends information regarding anyresponse back to the device 200.

The processor 210, touch-sensitive display 230, or other component mayuse the information received in any number of ways to determine whethera response is needed and, if so, what the response should be. Forexample, in one embodiment, the processor 210 may determine that animage associated with the touch-sensitive display 230 should be moved.In another embodiment, the touch-sensitive display 230 may determinethat the color of an object on the touch-sensitive display 230 should bechanged. In other embodiments, the processor 210 may determine that oneor more actuators need to output one or more haptic effects. Variousadditional responses are discussed below.

In some embodiments, duration data may be received by the processor 210,the touch-sensitive display 230, or the network interface 250 may beused to determine a response, if any. For example, in one embodiment,the processor 210 may determine a particular response if the length oftime that a contact has contacted the touch-sensitive display 230exceeds a threshold duration. In other embodiments, a response may bedetermined if the duration of a contact is below a threshold duration.The processor 210 may determine a response based upon the duration oftime of two or more contacts with the touch-sensitive display 230. Forexample, in one embodiment, if the duration of a first contact exceedsthe duration of a second contact, the processor 210 may determine aresponse. In other embodiments, a response may be determined if a secondcontact occurs within a predetermined time after a first contact withthe touch-sensitive display 230. For example, in one embodiment, asecond contact must be substantially simultaneous with a first contactfor the processor 210 to determine a response.

In some embodiments, location data may be received by the processor 210,the touch-sensitive display 230, or the network interface 250 may beused to determine a response, if any. The location of a contact may bedetermined in any number of ways. For example, the touch-sensitivedisplay 230 may be addressable using Cartesian x and y coordinates orpolar coordinates. Thus, in one embodiment, if the location of a firstcontact has an x coordinate that is larger than the x coordinate of thesecond location of a second contact, then the device 200 may determinethat the first location is greater than the second location. In anotherembodiment, if the location of a first contact has a y coordinate largerthan the y coordinate of the second location of a second contact, thenthe device 200 may determine that the first location is greater than thesecond location. Still in other embodiments, a formula based on the xand y coordinates of each contact may be used to determine the device's200 response, if any. For example, in one embodiment the formulasqrt(x²+y²) may be used to determine whether a contact is within aparticular area or distance from a specified location on thetouch-sensitive display 230. In another embodiment, the formula x+2y maybe used to determine whether a contact is within a rectangle from aspecified coordinate on the touch-sensitive display 230. In oneembodiment, the device 200 may determine the location of a contact bylogically dividing the touch-sensitive display 230 into sections. Forexample, the device 200 may logically divide a rectangulartouch-sensitive display 230 into three rows and three columns, thus,creating nine contact cells as shown in FIG. 4 as will be discussed inmore detail below.

Referring again to FIG. 4, a contact in section “B” and asubstantially-simultaneous contact in section “F” on the touch-sensitivedisplay 230 may cause the device 200 to determine that a response to thedetected contacts is to scroll a page displayed on the touch-sensitivedisplay 230 in a northeastern direction. Similarly, a contact in section“B” and a contact in section “D” may cause the device 200 to determinethat a response to the detected contacts is to scroll a page displayedon the touch-sensitive display 230 in a northwestern direction. In oneembodiment, the speed at which a page is scrolled on the touch-sensitivedisplay 230 is based on the pressure of one or more of the contacts or adifference in pressure between multiple contacts. The device 200 maydetermine that one or more haptic effects should be to alert a user thatan interaction has been detected, that a response is occurring, or thata response has been completed such as that the page is scrolling. In oneembodiment, the haptic effect may vary depending on the direction inwhich the page is scrolling.

For example, the processor 210 may determine that a haptic effect shouldbe output each time a contact is made with the touch-sensitive display230. Thus, as a user contacts sections “B” and “F,” the processor 210determines a haptic effect should be output in response to each contact.Further, once the contacts are recognized as a gesture, such as a scrollgesture, the processor 210 may determine a haptic effect associated withthe gesture.

In another embodiment, the processor 210 may determine that a responseto a detected contact or sequence of contacts is to update an imagedisplayed on the touch-sensitive display 230 and to output a hapticeffect. For example, a response may be that an image displayed on thetouch-sensitive display 230 is moved. In one embodiment, a response maybe that an image displayed on the touch-sensitive display 230 isrotated. For example, referring again to FIG. 4, if sections “B”, “D”,and “F” are contacted substantially simultaneously and then released,and then sections “B”, “D”, and “H” are contacted within a predeterminedperiod of time, then an image displayed on the touch-sensitive display230 may be rotated in a counter-clockwise direction. Similarly, ifsections “B”, “D”, and “F” are contacted substantially simultaneouslyand then released, and then sections “B”, “H”, and “F” are contactedwithin a predetermined period of time, such as 0.1 ms or 0.25 ms or 0.5ms, then an image displayed on the touch-sensitive display 230 may berotated in a clockwise direction. In some embodiments, a response may bethat at least one haptic effect is output to indicate that the image isbeing rotated in a clockwise or counter-clockwise direction on thetouch-sensitive display 230. For example, in one embodiment, theprocessor 210 may determine a haptic effect associated with a rotationof the image, such as a vibration that may be perceived to travel in thedirection of rotation or may increase in intensity or frequency thefarther the image is rotated. Alternatively, the processor 210 mayidentify a non-directional vibration effect that varies in frequencybased on the direction of rotation. For example, in one embodiment, thefrequency of the vibration may increase if the image is rotated in aclockwise direction or decrease if the image is rotated in acounterclockwise direction. Further, the processor may determine ahaptic effect, such as a pop or jolt, to be output in response to arotation of the image back to its starting orientation.

In one embodiment, a response may be that the graphics displayed on thetouch-sensitive display 230 are zoomed in or out. For example, referringstill to FIG. 4, if sections “A”, “C”, “G”, and “I” are contactedsubstantially simultaneously then the graphics displayed on thetouch-sensitive display 230 may be zoomed out. Similarly, if sections“B”, “D”, “F”, and “H” are contacted substantially simultaneously thenthe graphics displayed on the touch-sensitive display 230 may be zoomedin. In some embodiments, the processor 210 may determine that a responseis that one or more sounds needs to be output and can output thenecessary signals to the speaker 270. In other embodiments, a responsemay be that at least one haptic effect may be output to indicate thatthe graphics displayed on the touch-sensitive display 230 are beingzoomed out or zoomed in. For example, in one embodiment, the processor210 may determine a haptic effect associated with a level of zoom of theimage, such as a vibration that may increase in intensity or frequencythe greater the zoom, or may decrease in frequency or intensity thelower the zoom level. Further, the processor 210 may determine a hapticeffect, such as a pop or jolt, to be output in response to a rotation ofthe image back to its starting orientation.

In other embodiments, a response may be determined based on a change inlocation of one or more contacts on the touch-sensitive display 230. Forexample, the processor 210 may determine a response based on thelocation of a first contact changing in a northern direction and thelocation of a second contact changing in an eastern direction. Inanother embodiment, the processor 210 may determine a response based onthe location of a first contact moving in a western direction and asecond contact moving in an eastern direction. In other embodiments, theprocessor 210 can determine a response based on whether the location ofa first contact is moving in an opposite direction of the location of asecond contact on the touch-sensitive display 230.

In some embodiments, a response may be determined based on a specifiedinteraction with the device 200. An interaction can include any numberof actions based on one or more contacts. For example, in oneembodiment, the processor 210 may determine a response based on aninteraction where the interaction having a first contact having alocation corresponding with a graphical object on the touch-sensitivedisplay 230 and a second contact having a location not correspondingwith the graphical object on the touch-sensitive display 230. In otherembodiments, an interaction may be based two contacts having a locationcorresponding with a graphical object on the touch-sensitive display230. In various embodiments, an interaction may be based on twographical objects on the touch-sensitive display 230 where the locationof a first contact corresponds with the first graphical object and thelocation of a second contact corresponds with the second graphicalobject.

In other embodiments, the processor 210 can determine a response to acontact on the touch-sensitive display 230 based on a combination of thevarious data the processor 210 receives from the touch-sensitive display230 or the network interface 250 or one or more of the factors such as achange in location or an interaction. For example, in one embodiment, aresponse can be determined by the processor 210 based on both pressureand location of one or more contacts on the touch-sensitive display 230.In another embodiment, the processor 210 can determine a response basedon pressure and an interaction. For example, the processor 210 maydetermine that the color of a graphical displayed on the touch-sensitivedisplay 230 needs to be changed based upon a first contact having alocation corresponding to the graphical object and a second contact nothaving a location corresponding to the graphical object and the firstcontact having a specified pressure. Other embodiments are describedherein and still other embodiments would be apparent to one of skill inthe art.

Referring again to the embodiment shown in FIG. 3, once a response isdetermined as specified in block 330, the processor 210 generates asignal as specified in block 340. For example, in one embodimentdiscussed above, the processor 210 receives a signal from thetouch-sensitive display 230 when a user contacts the touch-sensitivedisplay 230 and the signal includes information associated with an inputon—or a status of—the touch-sensitive display 230 such as the x, ylocation and pressure of a contact on the touch-sensitive display 230.In this embodiment, if the user is viewing a web page displayed on thetouch-sensitive display 230 of the device 200 and if the processor 210determines that the user is touching the touch-sensitive display 230substantially simultaneously with two fingers (i.e. two contacts) and isapplying more pressure with a finger located nearer the bottom of thescreen than the other finger, the processor 210 determines that theresponse should be to update the touch-sensitive display 230 to scrolldown the web page and to output a haptic effect that indicates that thepage is scrolling down the web page. In this embodiment, the processor210 generates a first signal that is configured to cause thetouch-sensitive display 230 to scroll down the web page and theprocessor 210 generates a second signal that is configured to causeactuator 240 to output a haptic effect that indicates that the page isscrolling down the page.

In some embodiments, the processor 210 generates a single signal afterdetermining a response. For example, if the processor 210 determinesthat the touch-sensitive display 230 needs to be updated, then theprocessor 210 can generate a display signal and send the signal to thetouch-sensitive display 230 that causes the graphics associated with thetouch-sensitive display 230 to be updated. In other embodiments, theprocessor 210 generates two, three, or more signals. For example, in oneembodiment, the processor 210 generates a different signal for eachresponse that is determined in block 330 of the method 300 shown in FIG.3. Thus, if it is determined that the touch-sensitive display 230 needsto be updated, actuator 240 needs to output a first haptic effect, andactuator 260 needs to output a second haptic effect, then the processor210 may generate a first signal configured to cause the touch-sensitivedisplay 230 to be updated, a second signal configured to cause actuator240 to output a haptic effect, and a third signal configured to causeactuator 260 to output a haptic effect. In other embodiments, theprocessor 210 may generate one or more signals configured to cause thetouch-sensitive display 230, the network interface 250, the actuator240, the actuator 260, the speaker 270, other components of the device200, other components or devices in communication with the device 200,or a combination thereof to perform a particular function.

In one embodiment, a generated signal includes a command for a device orcomponent to perform a specified function, such as to output a hapticeffect, display an image, play a sound, or transmit a message to aremote device. In another embodiment, a generated signal includesparameters which are used by a device or component receiving the commandto determine a response or some aspect of a response. Parameters mayinclude various data related to, for example, magnitudes, frequencies,durations, or other parameters that an actuator can use to determine ahaptic effect, output a haptic effect, or both. For example, in oneembodiment, the processor 210 generates a signal configured to causeactuator 240 to output a haptic effect. In such an embodiment, thesignal may include a pressure parameter that the actuator 240 uses tothe intensity of the haptic effect to output. For example, according toone embodiment, the larger the pressure parameter the actuator 240receives, the more intense the haptic effect that is output. Thus, asignal may include data that is configured to be processed by anactuator, display, network interface, speaker, or other component of adevice or in communication with a device in order to determine an aspectof a particular response.

Referring again to FIG. 3, once a signal has been generated as specifiedin block 340, the next step of method 300 is to output the signal asshown in block 350. For example, in one embodiment discussed above, theprocessor 210 generated a first signal configured to cause thetouch-sensitive display 230 to scroll down the web page and theprocessor 210 generated a second signal configured to cause actuator 240to output a haptic effect that indicates that the page is scrolling downthe page. In such an embodiment, the processor 210 outputs the firstsignal to the touch-sensitive display 230 and outputs the second signalto actuator 240.

In various embodiments, the processor 210 may output one or moregenerated signals to any number of devices. For example, the processor210 may output one signal to the network interface 250. In oneembodiment, the processor 210 may output one generated signal to thetouch-sensitive display 230, another generated signal to the networkinterface 250, and another generated signal to the actuator 260. Inother embodiments, the processor 210 may output a single generatedsignal to multiple components or devices. For example, in oneembodiment, the processor 210 outputs one generated signal to bothactuator 240 and actuator 260. In another embodiment, the processor 210outputs one generated signal to actuator 240, actuator 260, and networkinterface 250. In still another embodiment, the processor 210 outputsone generated signal to both actuator 240 and actuator 260 and outputs asecond generated signal to the touch-sensitive display 230.

As discussed above, the processor 210 may output one or more signals tothe network interface 250. For example, the processor 210 may output asignal to the network interface 250 instructing the network interface250 to send data to a another component or device in communication withthe device 200. In such an embodiment, the network interface 250 maysend data to the other device and the other device may perform afunction such as updating a display associated with the other device orthe other device may output a haptic effect. Thus, in embodiments of thepresent invention, a second device may output a haptic effect based atleast in part upon an interaction with a first device in communicationwith the second device. In other embodiments, a second device mayperform any number of functions such as, for example, updating a displayassociated with the second device or outputting a sound to a speakerassociated with the second device based at least in part on aninteraction with a first multi-pressure touch-sensitive input device200.

In various embodiments, after the processor 210 outputs a signal to acomponent, the component may send the processor 210 a confirmationindicating that the component received the signal. For example, in oneembodiment, actuator 260 may receive a command from the processor 210 tooutput a haptic effect. Once actuator 260 receives the command, theactuator 260 may send a confirmation response to the processor 210 thatthe command was received by the actuator 260. In another embodiment, theprocessor 210 may receive completion data indicating that a componentnot only received an instruction but that the component has performed aresponse. For example, in one embodiment, actuator 240 may receivevarious parameters from the processor 210. Based on these parametersactuator 240 may output a haptic effect and send the processor 210completion data indicating that actuator 240 received the parameters andoutputted a haptic effect.

Another embodiment of the present invention that implements the method300 shown in FIG. 3 and that will be described with respect to thedevice shown in FIG. 2 is a paint mixing application. In thisembodiment, a user contacts the touch-sensitive display 230 with onefinger to select a first color and contacts the touch-sensitive display230 with a second finger to select a second color. In this embodiment,the touch-sensitive display 230 shows a third color that represents thefirst color being mixed with the second color. For example, if the firstcolor is red and the second color is yellow, then the third color shownon the touch-sensitive display 230 may be orange. In some embodiments,the shade of the third color may be changed by increasing or decreasingthe pressure of the contact of either the first or second finger. Thus,the amount of the first color shown in the third mixed color is afunction of the pressure of the first contact. Likewise, the amount ofthe second color shown in the third mixed color is a function of thepressure of the second contact. Thus, the third mixed color that isshown on the touch-sensitive display 230 is a function of the firstcolor, the first pressure, the second color, and the second pressure,thereby providing an intuitive mechanism for generating a new color tobe used. In various embodiments, one or more haptic effects are outputbased at least in part on an applied pressure of at least one contact.

An embodiment of a sculpting application that implements the method 300shown in FIG. 3 is described below with respect to the device shown inFIG. 2. In one embodiment, a piece of clay is displayed on thetouch-sensitive display 230. A user can interact with and shape the clayby contacting the touch-sensitive display 230. For example, if a usercontacts the touch-sensitive display 230 with three fingers, eachcorresponding with a different location on the clay, then thetouch-sensitive display 230 is updated to show the clay with adeformation at each location. The user may further be able to performmulti-touch, multi-pressure gestures to further shape the clay in afashion similar to how actual clay may be manipulated. Thus, a user isable to shape and form the clay on the touch-sensitive display 230. Insome embodiments, the rate of deformation of a contact may be a functionof the pressure or movement speed that a user is applying with eachfinger. In various embodiments, one or more haptic effects are outputbased at least in part on applied pressure of a contact.

One embodiment of the present invention is directed to a texture-basedapplication that implements the method 300 shown in FIG. 3. Thisembodiment will be described with respect to the device shown in FIG. 2.In this embodiment, an image representing one or more textures isdisplayed on the touch-sensitive display 230. A user can contact thetouch-sensitive display with one or more fingers and drag one or morefingers across the image which represents a texture and, in response,one or more haptic effects may be output. For example, in oneembodiment, an image of a piece of sandpaper is displayed on thetouch-sensitive display 230. In this embodiment, a user can contact thetouch-sensitive display 230 with a finger and move the finger alongportions of the display 230 where the image of the piece of sandpaper islocated. In response, one or more haptic effects may be output thatindicate the texture of the image such that the haptic effects simulatethe feeling of actually rubbing a finger along a piece of sandpaper. Insome embodiments, haptic effects may be output that are based on thepressure of each contact. Thus, if a user increases the pressure of acontact on the touch-sensitive display 230 then the magnitude of one ormore haptic effects may be increased as well or a friction between theuser's fingers and the touch-sensitive surface may be increased, such asby changing a shape of the touch-sensitive surface or by raisingfeatures on the touch-sensitive surface. Therefore, in embodiments, oneor more haptic effects that simulate the texture of one or more objectsdisplayed on the touch-sensitive display 230 may be output in responseto multi-touch, multi-pressure contacts or gestures.

In a further embodiment, an image of a keyboard is displayed on thetouch-sensitive display 230. A user can interact with the device bycontacting the touch-sensitive display 230 at locations which correspondto keys on the keyboard. In some embodiments, a user may use multiplefingers to type on the keyboard. In this embodiment, a haptic effect maybe output based on the pressure of one or more contacts. For example, inone embodiment, the magnitude of a haptic effect is a function of thepressure in which a user contacts the touch-sensitive display 230. Thus,the harder (i.e. more pressure) that a user contacts the touch-sensitivedisplay 230, the stronger the haptic effect.

While the steps of method 300 have been shown and described in aparticular order, other embodiments of the present invention maycomprise the same or additional steps or may perform the steps shown inFIG. 3 in a different order or in parallel. For example, the method mayreceive a plurality of sensor signals and determine a plurality ofcontacts substantially simultaneously or in succession prior todetermining a response.

Illustrative Method of Detecting and Responding to a Contact

Referring now to FIG. 5, FIG. 5 illustrates a flow chart directed to amethod 500 of outputting an actuator signal in an multi-pressuretouch-sensitive input device in accordance with an embodiment of thepresent invention. The description of the method 500 of FIG. 5 will bemade with respect to the device 200 shown in FIG. 2 and the exampleshown in FIG. 6.

FIGS. 6A-6C illustrate the operation of a multi-pressure touch-sensitiveinput device 200 in accordance with an embodiment of the presentinvention. Embodiments of the present invention may allow a user tointeract with an object using multi-contact, multi-pressure inputs. Insome embodiments, an object may be a graphical user object, such as abutton, a scroll bar, a radio button, etc. In some embodiments, anobject may be any graphical object or textual object displayed on ascreen. For example, in FIG. 6A, a graphical object is a seesaw 610having a horizontal board 620 is displayed on the touch-sensitivedisplay 230 of a multi-pressure touch-sensitive input device 200. A userinteracts with the seesaw 610 by contacting the touch-sensitive display230 at various locations and with various pressures.

The method 500 shown in FIG. 5 begins in block 510 when the processor210 receives first and second sensor signals. For example, as a userinteracts with the device 200, the processor 210 is provided withinformation, such as a first pressure, related to a first contact if theuser contacts a location on the touch-sensitive display 230 associatedwith the left side of the seesaw 610. In addition, as the user interactswith the device 200, the processor 210 is provided with information,such as a second pressure, related to a second contact if the usercontacts a location on the touch-sensitive display 230 associated withthe right side of the seesaw 610. After receiving the sensor signals,the method 500 proceeds to block 520.

In block 520, the processor 210 determines whether the pressure of thefirst contact is greater than the pressure of the second contact 520. Ifthe pressure of the first contact is greater than the pressure of thesecond contact, the method 500 proceeds to block 530, otherwise itproceeds to block 550.

In block 530, the processor 210 generates a first actuator signal. Inthe embodiment shown in FIG. 6, the first actuator signal is configuredto cause actuator 240 to output a haptic effect that simulates the leftside of the board 620 being moved down because the first pressure isgreater than the second pressure. The generated signal can comprise someor all of the data, instructions, or parameters discussed above withrespect to the embodiment shown in FIG. 3. For example, in oneembodiment, the generated signal includes a parameter that the firstactuator uses to determine the intensity of the haptic effect based onthe greater pressure. In another embodiment, the generated actuatorsignal may include a parameter based on the difference in pressuresbetween a contact on the left side of the seesaw 620 and a contact onthe right side of the seesaw 620. In still other embodiments, thegenerated actuator signal may include information related to how closethe left side of the seesaw 620 is to the ground. In the embodimentshown in FIG. 6, the processor 210 generates another signal configuredto cause the touch-sensitive display 230 to update the image on thedisplay as shown in FIG. 6B.

Once the processor 210 generates the first actuator signal as shown inblock 530, the processor 210 outputs the first actuator signal as shownin block 540. For example, in the embodiment shown in FIG. 6, theprocessor 210 outputs the actuator signal to actuator 240. In responseto receiving the signal from the processor 210, the actuator 240 outputsthe desired haptic effect. In the embodiment shown in FIG. 6, theprocessor 210 also outputs another signal which causes the display to beupdated as shown in FIG. 6B.

In block 550, the processor 210 generates a second actuator signal andoutputs the second actuator signal 560 to actuator 260. In thisembodiment, the second actuator signal includes a magnitude parameterthat actuator 260 uses to determine the desired haptic effect andactuator 260 outputs the haptic effect. For example, in the embodimentshown in FIG. 6, if the first pressure is not greater than the secondpressure, then the processor 210 generates a second actuator signalconfigured to cause actuator 240 to output a haptic effect thatsimulates the right side of the board 620 being moved down because thefirst pressure is not greater than the second pressure. In addition, inthis embodiment, the processor 210 generates another signal configuredto cause the touch-sensitive display 230 to update the image on thedisplay as shown in FIG. 6C. After the processor 210 generates thesecond actuator signal and the other signal, the processor 210 outputsthe second actuator signal to the actuator 260 and outputs the othersignal to the touch-sensitive display 230. In response, the actuator 260outputs the desired haptic effect and the display updates the display asshown in FIG. 6C.

Thus, in the embodiment shown in FIG. 6, the pressure of the contacts oneach side of the board 620 may be correlated to forces being applied toeach side of the board 620. If the pressure, i.e. a simulated force inthis embodiment, is greater on one side of the board 620 than on theother side of the board 620, then the touch-sensitive display 230updates to indicate that the board 620 tilts in favor of the side underthe greater simulated force and an actuator outputs a haptic effectindicating the board 620 is tilting in a corresponding direction. In oneembodiment, the degree to which the side with the greater simulatedforce tilts depends upon the difference in pressures of the contacts.Thus, the board 620 of the seesaw 610 is not tilted as much in FIG. 6Cas it is in FIG. 6B because the difference in pressures is not as largein FIG. 6C as it is in FIG. 6B. Furthermore, in such an embodiment, thehaptic effect output in FIG. 6B may be more intense than the hapticeffect output in FIG. 6C because the difference in pressures is not aslarge in FIG. 6C as it is in FIG. 6B.

Another embodiment of the present invention that implements the method500 shown in FIG. 5 and that will be described with respect to thedevice shown in FIG. 2 is a snowboarding application. In thisembodiment, a user can interact with the device 200 by contacting thetouch-sensitive display 230 with two fingers. The pressure of a contactmay be used to steer the snowboarder. For example, if the snowboarder isinitially shown in the middle of the touch-sensitive display 230 whenthe user's finger on the left side of the screen increases its contactpressure, the snowboarder moves to the left. Or the snowboarder may moveto the right if the pressure of a contact on the right side of thescreen is increases or is greater than the pressure of a contact on theleft side of the screen.

In some embodiments, the pressure of one or more contacts may be used todetermine the rate of turning. For example, in one embodiment anincrease in pressure of a contact results in an increased rate ofturning. In other embodiments, the pressure of one or more contacts isused to determine both direction and rate of turning. For example, in anembodiment, the pressure of one contact determines the direction of thesnowboarder (i.e. left or right) and a pressure of another contactdetermines the rate of turning. In this embodiment, the direction of thesnowboarder may be a function of a threshold pressure. Thus, if thepressure of the contact associated with the direction of the snowboarderis greater than the threshold pressure, the snowboarder may move to theright. If the pressure of the contact associated with the direction ofthe snowboarder is less than the threshold pressure, the snowboarder maymove to the left. Furthermore, in this embodiment, the rate of turningmay be a function of the pressure. Thus, an increase in pressure of thecontact associated with the rate of turning may result in an increase inthe rate of turning of the snowboarder. Likewise, a decrease in pressureof the contact associated with the rate of turning may result in adecrease in the rate of turning of the snowboarder.

In embodiments, one or more haptic effects may also be output based atleast in part on the pressure of one or more of the contacts to indicateto the user the direction or the rate of turning, or both. For example,in one embodiment, a haptic effect may be output that indicates that thesnowboarder is moving to the left and another haptic effect may beoutput that indicates that the snowboarder is moving to the right. Forexample, a vibration may be output on a right side of the device, or avibration may be output on a left side of the device and move to theright side of the device at a rate corresponding to the rate of thesnowboarder's turning. In another embodiment, a haptic effect may beoutput that indicates that the rate of turning of the snowboarder isincreasing and another haptic effect may be output that indicates thatthe rate of turning of the snowboarder is decreasing, such as byincreasing or decreasing a frequency or magnitude of a vibration.

Illustrative Method of Detecting and Responding to a Contact

Referring now to FIG. 7, FIG. 7 illustrates the operation of amulti-pressure touch-sensitive input device 200 as shown in FIG. 2 inaccordance with an embodiment of the present invention. With respect tothe embodiment shown in FIG. 7, the method of FIG. 5 may be employed toprovide haptic feedback as a user plays a virtual keyboard 710. In theembodiment shown in FIG. 7, the keys of a piano are displayed on atouch-sensitive display 230. In response to a user touching a locationof the touch-sensitive display 230 corresponding with the “C” note ofthe keyboard, the device 200 outputs a sound having a frequencycorresponding with the “C” note by generating an audio signal andtransmitting the audio signal the speaker 270. Likewise, in response toa user touching locations on the touch-sensitive display 230corresponding with notes “C”, “E”, and “G” substantially simultaneously,the device 200 outputs a sound having a frequency corresponding with the“C-E-G” chord by generating a different audio signal and transmitting itto the speaker 270.

In another embodiment, a user may touch locations on the touch-sensitivedisplay 230 with one hand corresponding with notes “C”, “E”, and “G” andsubstantially simultaneously the user may touch locations on thetouch-sensitive display 230 with another hand corresponding with notes“D”, “F”, and “A”. In response, the device 200 may output a sound havinga frequency corresponding with the “C-E-G” chord and a sound having afrequency corresponding with the “D-F-A” chord. In some embodiments, thedevice 200 may output one or more haptic effects to alert the user thata particular chord or combination of chords, or both, is being pressedby the user. For example, one or more haptic effects may be output thatindicate which chord is being played. In such an embodiment, one hapticeffect is output if a user plays the “C-E-G” chord and a differenthaptic effect is output if a user plays the “D-F-A” chord. Thus, ahearing impaired user or a user that wants sound on the device to bemuted, can practice playing the simulated piano 710 and determine whichchords are being played based upon one or more haptic effects output bythe device 200. In another embodiment, the intensity of one or morehaptic effects output by the device 200 may be increased or decreased asa user increases or decreases, respectfully, the pressure on variouscontacts on the simulated keyboard 710. Thus, a user can simulateplaying a keyboard by pressing locations on the touch-sensitive display230 corresponding with the various notes that the user wishes to playand can receive haptic feedback indicating the note or notes that theuser presses.

In one embodiment, the processor 210 executes software that determineswhether the user is playing the correct notes at the correct time for agiven song. For example, for a particular song the notes “C” and “E” mayneed to be played simultaneously followed by the notes “D”, “F”, and “A”played simultaneously. If the user incorrectly presses notes “C” and “F”instead of notes “C” and “E” the device 200 may output a haptic effectalerting the user that an incorrect note has been played. Likewise, ifthe user correctly plays notes “C” and “E” simultaneously and playsnotes “D”, “F”, and “A” simultaneously but with an incorrect timing,(i.e. the notes are played too fast or too slowly), the device 200 mayoutput a different haptic effect alerting the user that their timing wasincorrect.

In another embodiment, a first multi-pressure touch-sensitive inputdevice 200 is in communication with a second multi-pressuretouch-sensitive input device 200. In this embodiment, thetouch-sensitive display 230 of the first device 200 may display the sameinformation as the touch-sensitive display 230 of the second device 200.For example, both devices may display a keyboard as shown in FIG. 7. Theprocessor 210 of the first device 200 and the processor 210 of thesecond device 200 may execute software on the memory 220 of eachrespective device such that the user interacting with the first device200 is supposed to play one portion of a song and another userinteracting with the second device 200 is supposed to play anotherportion of a song. In one embodiment, if the first user incorrectlyplays a note on the first device 200, then a haptic effect is output bythe first device 200. In another embodiment, if the first userincorrectly plays a note on the first device 200, the first device sendsa command or instruction to a second device to output a haptic effectand the second device outputs a haptic effect. In yet anotherembodiment, if the first user incorrectly plays a note on the firstdevice 200, the first device 200 sends data to the second device 200regarding the incorrect note that the first user played and the seconddevice 200 determines whether a haptic effect, if any, needs to beoutput on the second device 200 or on first device 200, or both.

Illustrative Method of Detecting and Responding to a Contact

Referring now to FIG. 8, FIG. 8 illustrates a flow chart directed to amethod 800 of outputting an actuator signal in a multi-pressuretouch-sensitive input device 200 in accordance with an embodiment of thepresent invention. The description of the method 800 of FIG. 8 will bemade with respect to the device 200 shown in FIG. 2 and the exampleshown in FIG. 9.

The method shown in FIG. 8 begins in block 810 when a graphical objectis displayed. For example, in FIG. 9, a graphical object 910 isdisplayed on the touch-sensitive display 230. After displaying thegraphical object, the method 800 proceeds to block 820.

In block 820, a first contact and a second contact are received. Forexample, in the embodiment shown in FIG. 9, a user can interact with thedevice 200 by contacting the touch-sensitive display 230 with a finger.In this embodiment, as the user interacts with the device 200, theprocessor 210 is provided with information, such as a first x, ycoordinate and a first pressure, associated with a first contact from afirst finger on the touch-sensitive display 230. In addition, as theuser interacts with the device 200, the processor 210 is provided withinformation, such as a second x, y coordinate and a second pressure,associated with a second contact from a second finger on thetouch-sensitive display 230. In some embodiments, the first contact andthe second contact on the touch-sensitive display may need to occursubstantially simultaneously in order for the processor 210 to receiveinformation a first and a second contact.

Once a first contact and a second contact are received, the method 800proceeds to block 830. In block 830, a determination is made as towhether the first contact is in a location corresponding to thegraphical object. For example, in the embodiment shown in FIG. 9, if thelocation of the first contact corresponds to a location where thegraphical object 910 is displayed on the touch-sensitive display 230,then the processor 210 determines that the first contact is a contact onthe graphical object. However, in this embodiment, if the location ofthe first contact does not correspond to a location where the graphicalobject 910 is displayed on the touch-sensitive display 230, then theprocessor 210 determines that the first contact is not on the graphicalobject. If the first contact is on the graphical object, then the method800 proceeds to block 840; otherwise, it proceeds to block 865.

In block 840, a determination is made as to whether the second contactis in a location corresponding to the graphical object. For example, inFIG. 9, if the location of the second contact corresponds to a locationwhere the graphical object 910 is displayed on the touch-sensitivedisplay 230, then the processor 210 determines that the second contactis a contact on the graphical object. However, in this embodiment, ifthe location of the second contact does not correspond to a locationwhere the graphical object 910 is displayed on the touch-sensitivedisplay 230, then the processor 210 determines that the second contactis not on the graphical object. If the second contact is on thegraphical object, then the method 800 proceeds to block 845; otherwise,it proceeds to block 855.

In block 845, the processor 210 generates a first actuator signal. Forexample, in FIG. 9, if the first contact is on the graphical object 910and the second contact is also on the graphical object 910, then theprocessor 210 determines that a response is to enlarge the size of thegraphical object 910 displayed on the touch-sensitive display 230. Inaddition, in this embodiment, the processor 210 determines that aresponse is to output a haptic effect that indicates that the size ofthe graphical object 910 displayed on the touch-sensitive display 230 isbeing enlarged. In this embodiment, the processor 210 generates a firstactuator signal configured to cause actuator 240 to output a hapticeffect that indicates that the size of the graphical object 910displayed on the touch-sensitive display 230 is being enlarged, such asa increasing-frequency or intensity vibration. In addition, theprocessor 210 may generate a first actuator signal that also comprises ahaptic effect to indicate the two contacts on the object, such as a popor jolt. In addition, processor 210 generates a display signalconfigured to cause the touch-sensitive display 230 to enlarge the sizeof the graphical object 910 displayed on the touch-sensitive display230.

Referring still to FIG. 9 and in reference to block 845, in oneembodiment, the processor 210 determines that if the pressure of thefirst contact is greater than a threshold pressure then the size of thegraphical object 910 displayed on the touch-sensitive display 230 needsto be enlarged. Otherwise, in this embodiment, the processor 210 maydetermine that no response is needed. In another embodiment, theprocessor 210 determines that the size of the graphical object 910displayed on the touch-sensitive display 230 needs to be enlarged untilthe pressure of the second contact is below a threshold pressure.

Once the first actuator signal has been generated as shown in block 845,the processor 210 outputs the first actuator signal as shown in block850. For example, in the embodiment shown in FIG. 9, the processor 210outputs the generated first actuator signal to actuator 240. Actuator240 receives the first actuator signal from the processor 210 andoutputs a haptic effect indicating that the size of the graphical object910 displayed on the touch-sensitive display 230 is being enlarged. Inaddition, in the embodiment shown in FIG. 9, the processor 210 outputsthe generated display signal to the touch-sensitive display 230 and thetouch-sensitive display 230 updates the size of the graphical object 910shown on the touch-sensitive display 230.

In block 855, the processor 210 generates a second actuator signal. Forexample, in FIG. 9, if the first contact is on the graphical object 910and the second contact is not on the graphical object 910, then theprocessor 210 determines that a response is to change the color of thegraphical object 910. In addition, in this embodiment, the processor 210determines that a response is to output a haptic effect that indicatesthat the color of the graphical object 910 displayed on thetouch-sensitive display 230 is changing. For example, the processor 210may determine that a pop or jolt effect should be output each time thecolor is changed. In this embodiment, the processor 210 generates asecond actuator signal configured to cause actuator 260 to output ahaptic effect indicating that the color of the graphical object 910displayed on the touch-sensitive display 230 is changing. In addition,the processor 210 generates a display signal configured to cause thetouch-sensitive display 230 to change the color of the graphical object910 displayed on the touch-sensitive display 230.

Referring still to FIG. 9 and in reference to block 855, in oneembodiment, the processor 210 determines that if the pressure of thefirst contact is greater than a threshold pressure then the color of thegraphical object 910 displayed on the touch-sensitive display 230 needsto be changed. Otherwise, in this embodiment, the processor 210 maydetermine that no response is needed. In another embodiment, theprocessor 210 determines that the color of the graphical object 910displayed on the touch-sensitive display 230 needs to be changed untilthe pressure of the second contact is below a threshold pressure. Forexample, the color of the graphical object 910 may change atpredetermined time intervals from yellow to green to blue until thepressure of the second contact is below a threshold pressure. In oneembodiment, the color of the graphical object 910 changes based on thepressure. For example, the color of the graphical object 910 may changefrom red to yellow to green to blue as the pressure of the first contactincrease.

Once the second actuator signal has been generated as shown in block855, the processor 210 outputs the second actuator signal as shown inblock 860. For example, in the embodiment shown in FIG. 9, the processor210 outputs the generated second actuator signal to actuator 260.Actuator 260 receives the second actuator signal from the processor 210and outputs a haptic effect indicating that the color of the graphicalobject 910 displayed on the touch-sensitive display 230 is beingchanged. In addition, in the embodiment shown in FIG. 9, the processor210 outputs the generated display signal to the touch-sensitive display230 and the touch-sensitive display 230 updates the color of thegraphical object 910 shown on the touch-sensitive display 230.

If it was determined in block 830 that the first contact was not in alocation corresponding to the graphical object, the method proceeds toblock 865. In block 865, a determination is made as to whether thesecond contact is in a location corresponding to the graphical object.For example, in FIG. 9, if the location of the second contactcorresponds to a location where the graphical object 910 is displayed onthe touch-sensitive display 230, then the processor 210 determines thatthe second contact is a contact on the graphical object. However, inthis embodiment, if the location of the second contact does notcorrespond to a location where the graphical object 910 is displayed onthe touch-sensitive display 230, then the processor 210 determines thatthe second contact is not on the graphical object. If the second contactis on the graphical object, then the method 800 proceeds to block 870;otherwise, it proceeds to block 880.

In block 870, the processor 210 generates a third actuator signal. Forexample, in FIG. 9, if the first contact is not on the graphical object910 and the second contact is on the graphical object 910, then theprocessor 210 determines that a response is to move the location ofwhere the graphical object 810 is displayed on the touch-sensitivedisplay 230. In addition, in this embodiment, the processor 210determines that a response is to output a haptic effect that indicatesthat the location of the graphical object 910 displayed on thetouch-sensitive display 230 is changing. In this embodiment, theprocessor 210 generates a third actuator signal configured to causeactuator 240 to output a haptic effect indicating that the location ofthe graphical object 910 displayed on the touch-sensitive display 230 ischanging. In addition, the processor 210 generates a display signalconfigured to cause the touch-sensitive display 230 to change thelocation of where the graphical object 910 is displayed on thetouch-sensitive display 230.

Referring still to FIG. 9 and in reference to block 870, in oneembodiment, the processor 210 determines that if the pressure of thefirst contact is greater than a threshold pressure, then a response isto move the graphical object 910 in an upward direction. If the pressureof the first contact is less than the threshold pressure, then aresponse is to move the graphical object 910 in a downward direction. Insome embodiments, if the pressure of the second contact is greater thana threshold pressure, then a response is to move the graphical object tothe left. If the pressure of the second contact is less than thethreshold pressure, then a response is to move the graphical object 910to the right. In some embodiments, a response may be determined basedupon both the first pressure and the second pressure. For example, inone embodiment, a response may be to move location of the graphicalobject 910 both upwards and to the left based upon the first pressureand the second pressure. In another embodiment, the location of thegraphical object 910 may be changed based upon a change in location ofthe contact. Thus, if the location of the first contact moves in anupward direction, the location of the graphical object 910 displayed onthe touch-sensitive display 230 may also be moved in an upwarddirection. In one embodiment, both the location and the color or size ofthe graphical object 910 displayed on the touch-sensitive display 230may be changed based upon the location and pressure of both the firstcontact and the second contact.

Once the third actuator signal has been generated as shown in block 870,the processor 210 outputs the third actuator signal as shown in block875. For example, some embodiments disclosed above, the processor 210outputs the generated third actuator signal to actuator 240. Actuator240 receives the third actuator signal from the processor 210 andoutputs a haptic effect indicating that the location of where thegraphical object 910 is displayed on the touch-sensitive display 230 isbeing changed. In addition, embodiments shown with respect to FIG. 9,the processor 210 outputs the generated display signal to thetouch-sensitive display 230 and the touch-sensitive display 230 updatesthe location of the graphical object 910 shown on the touch-sensitivedisplay 230.

In block 880, the processor 210 generates a fourth actuator signal. Forexample, in FIG. 9, if the first contact is not on the graphical object910 and the second contact is not on the graphical object 910, then theprocessor 210 determines that a response is to reduce the size of thegraphical object 910. In addition, in this embodiment, the processor 210determines that a response is to output a haptic effect that indicatesthat the size of the graphical object 910 displayed on thetouch-sensitive display 230 is being reduced. In this embodiment, theprocessor 210 generates a fourth actuator signal configured to causeactuator 260 to output a haptic effect indicating that the size of thegraphical object 910 displayed on the touch-sensitive display 230 isbeing reduced, such as a decreasing-frequency or intensity vibration. Inaddition, the processor 210 generates a display signal configured tocause the touch-sensitive display 230 to reduce the size of thegraphical object 910 displayed on the touch-sensitive display 230.

Referring still to FIG. 9 and in reference to block 880, in oneembodiment, the processor 210 determines that if the change in locationof the first contact is in a rightward direction and the change inlocation of the second contact is in a leftward direction, then the sizeof the graphical object 910 displayed on the touch-sensitive display 230needs to be reduced. In another embodiment, the processor 210 determinesthat if the change in location of the first contact is in an oppositedirection of the change in location of the second contact and thelocations of the first and second contacts are changing such that thefirst contact and second contact are coming closer together, then thesize of the graphical object 910 displayed on the touch-sensitivedisplay 230 needs to be reduced. In one embodiment, in addition toreducing the size of the graphical object 910 displayed on thetouch-sensitive display 230 the processor may determine a response thatthe color of the graphical object 910 needs to be changed or that thelocation of the graphical object 910 needs to move based upon thepressure of the first contact, the pressure of the second contact, orboth.

Once the fourth actuator signal has been generated as shown in block880, the processor 210 outputs the fourth actuator signal as shown inblock 885. For example, in the some embodiments discussed above withrespect to FIG. 9, the processor 210 outputs the generated fourthactuator signal to actuator 260. Actuator 260 receives the fourthactuator signal from the processor 210 and outputs a haptic effectindicating that the size of the graphical object 910 displayed on thetouch-sensitive display 230 is being reduced. In addition, inembodiments shown in FIG. 9, the processor 210 outputs the generateddisplay signal to the touch-sensitive display 230 and thetouch-sensitive display 230 updates the size of the graphical object 910shown on the touch-sensitive display 230.

Another embodiment of the present invention that implements the method800 shown in FIG. 8 and that will be described with respect to thedevice shown in FIG. 2 is a networked application. In this embodiment,two multi-pressure touch-sensitive input devices 200 are incommunication with each other using respective network interfaces 250.For example, in one embodiment the devices 200 communicate with eachother over the Internet. In another embodiment, the communication may beover a wireless network.

In various embodiments, one or more haptic effects may be output basedon two or more contacts on one device and two or more contacts onanother device. For example, two devices 200 may be in communicationwith each other and a user of one device 200 may touch a first locationon the display 230 with a first finger and may touch a second locationon the display 230 with a second finger. Likewise, a user of the seconddevice 200 may touch a first location on the display 230 with a firstfinger and may touch a second location on the display 230 with a secondfinger. In one embodiment, the location of the first contact on thefirst device substantially corresponds with the location of the firstcontact on the second device and the location of the second contact onthe first device substantially corresponds with the location of thesecond contact on the second device, then a response may occur. Forexample, in one embodiment, the response may be that access is grantedto either or both users to a file, website, application, etc. Inembodiments, the response may include one or more haptic effectsindicating that access is granted or that the locations of both contactson each device are substantially at the same location. In otherembodiments, one or more haptic effects may be output to either deviceor both devices indicating that at least one of the contacts does notmatch if any of the contacts are not at a substantially similarlocation.

In some embodiments, one or more haptic effects may be output based onthe pressure of a contact on a first device and a pressure of a contacton a second device where the first device and the second device are incommunication with each other. For example, in a wrestling applicationwhere two or more devices are in communication with each other, a userof one of the devices may contact the touch-sensitive display 230 at onelocation and with a first pressure. A user of another device may contactthe touch-sensitive display 230 at a second location corresponding withthe first location on the display of the first device and with a secondpressure. In this embodiment, one or more haptic effects may be outputon either device or both devices based on the pressure of the contacts.For example, in one embodiment, if the pressure of the first contact onthe first device is greater than the pressure of the second contact onthe second device, then a haptic effect may be output on the seconddevice indicating that the first user is punching harder than the seconduser. In another embodiment, if the pressure of the second contact onthe second device is greater than the pressure of the first contact onthe first device, then a haptic effect may be output on the first deviceindicating that the second user is pushing or grappling harder than thefirst user and another haptic effect may be output on the second deviceindicating that the second user is currently winning the match.

In one embodiment, one or more haptic effects may be output to a devicein communication with another device. For example, a first device suchas a touch-sensitive mobile phone may be in communication with a seconddevice such as a wearable arm strap that has one or more actuators. Inresponse to various multi-touch, multi-pressure interactions on thetouch-sensitive mobile phone, the phone may send one or more actuationcommands to the arm strap. In the embodiment, the arm strap receives oneor more of the actuation commands and, in response, outputs one or morehaptic effects. Thus, in embodiments, a haptic effect generation devicemay be used to output one or more haptic effects. The haptic effectgeneration device can be separate from a touch-sensitive device whichreceives multi-touch, multi-pressure contacts.

General

While the methods and systems herein are described in terms of softwareexecuting on various machines, the methods and systems may also beimplemented as specifically-configured hardware, such afield-programmable gate array (FPGA) specifically to execute the variousmethods. For example, referring again to FIGS. 1-2, embodiments can beimplemented in digital electronic circuitry, or in computer hardware,firmware, software, or in a combination of thereof. In one embodiment, adevice may comprise a processor or processors. The processor comprises acomputer-readable medium, such as a random access memory (RAM) coupledto the processor. The processor executes computer-executable programinstructions stored in memory, such as executing one or more computerprograms for editing an image. Such processors may comprise amicroprocessor, a digital signal processor (DSP), anapplication-specific integrated circuit (ASIC), field programmable gatearrays (FPGAs), and state machines. Such processors may further compriseprogrammable electronic devices such as PLCs, programmable interruptcontrollers (PICS), programmable logic devices (PLDs), programmableread-only memories (PROMs), electronically programmable read-onlymemories (EPROMs or EEPROMs), or other similar devices.

Such processors may comprise, or may be in communication with, media,for example computer-readable media, that may store instructions that,when executed by the processor, can cause the processor to perform thesteps described herein as carried out, or assisted, by a processor.Embodiments of computer-readable media may comprise, but are not limitedto, an electronic, optical, magnetic, or other storage device capable ofproviding a processor, such as the processor in a web server, withcomputer-readable instructions. Other examples of media comprise, butare not limited to, a floppy disk, CD-ROM, magnetic disk, memory chip,ROM, RAM, ASIC, configured processor, all optical media, all magnetictape or other magnetic media, or any other medium from which a computerprocessor can read. The processor, and the processing, described may bein one or more structures, and may be dispersed through one or morestructures. The processor may comprise code for carrying out one or moreof the methods (or parts of methods) described herein.

The foregoing description of some embodiments of the invention has beenpresented only for the purpose of illustration and description and isnot intended to be exhaustive or to limit the invention to the preciseforms disclosed. Numerous modifications and adaptations thereof will beapparent to those skilled in the art without departing from the spiritand scope of the invention.

Reference herein to “one embodiment” or “an embodiment” means that aparticular feature, structure, operation, or other characteristicdescribed in connection with the embodiment may be included in at leastone implementation of the invention. The invention is not restricted tothe particular embodiments described as such. The appearance of thephrase “in one embodiment” or “in an embodiment” in various places inthe specification does not necessarily refer to the same embodiment. Anyparticular feature, structure, operation, or other characteristicdescribed in this specification in relation to “one embodiment” may becombined with other features, structures, operations, or othercharacteristics described in respect of any other embodiment.

That which is claimed is:
 1. A method, comprising: receiving a firstsensor signal from a touch-sensitive input device in response to a firstcontact on the touch-sensitive input device, the first sensor signalcomprising a first location of the first contact and a first pressure ofthe first contact; receiving a second sensor signal from thetouch-sensitive input device in response to a second contact on thetouch-sensitive input device substantially simultaneously with the firstcontact, the second sensor signal comprising a second location of thesecond contact and a second pressure of the second contact; updating adisplay corresponding to the touch-sensitive input device based at leastin part on the first pressure being different than the second pressure;generating a signal configured to cause a haptic effect indicating theupdating of the display; and outputting the signal by wirelessly sendingthe signal to an arm strap comprising a haptic output device configuredto output the haptic effect.
 2. The method of claim 1, wherein thehaptic output device is an actuator and the signal comprises an actuatorsignal configured to cause the actuator to output the haptic effect. 3.The method of claim 2, wherein the actuator is configured to output thehaptic effect to the touch-sensitive input device.
 4. The method ofclaim 3, wherein the actuator comprises a piezo-electric actuator, arotary motor, or a linear resonant actuator.
 5. The method of claim 2,wherein the actuator comprises a plurality of actuators.
 6. The methodof claim 1, wherein the signal is based at least in part on a firstpressure threshold corresponding to the first pressure of the firstcontact.
 7. The method of claim 6, wherein the signal is based at leastin part on a second pressure threshold corresponding to the secondpressure of the second contact.
 8. The method of claim 1, wherein thesignal is based at least in part on a difference between the firstpressure of the first contact and the second pressure of the secondcontact.
 9. The method of claim 1, wherein the haptic effect comprisesat least one of a vibration, a friction, a texture, or a deformation.10. The method of claim 1, wherein a portable electronic devicecomprises a touch-sensitive display comprising the touch-sensitive inputdevice and the display, and wherein a graphical object is displayed onthe display prior to receiving the first sensor signal and prior toreceiving the second sensor signal.
 11. The method of claim 10, whereinupdating the display comprises enlarging at least part of the graphicalobject in response to a determination that the first pressure of thefirst contact is greater than a first threshold pressure and continuingto enlarge the at least part of the graphical object until the secondpressure of the second contact is below a second threshold pressure. 12.The method of claim 10, wherein updating the display comprises changingcolors of at least part of the graphical object in response to adetermination that the first pressure of the first contact is greaterthan a first threshold pressure and continuing to change colors of theat least part of the graphical object until the second pressure of thesecond contact is below a second threshold pressure.
 13. The method ofclaim 12, wherein continuing to change colors comprising changing colorsat predetermined time intervals.
 14. The method of claim 12, whereincontinuing to change colors comprises changing colors as a pressure ofat least one of the first pressure or the second pressure increases. 15.The method of claim 10, wherein the haptic effect is output responsiveto at least one of the first location of the first contact or the secondlocation of the second contact corresponding to a same location as thegraphical object displayed on the display.
 16. The method of claim 10,wherein the haptic effect is output responsive to both the firstlocation of the first contact and the second location of the secondcontact corresponding to a same location as the graphical objectdisplayed on the display.
 17. The method of claim 10, wherein updatingthe display comprises deforming at least part of the graphical objectbased on the first location of the first contact, the first pressure ofthe first contact, the second location of the second contact, and thesecond pressure of the second contact.
 18. The method of claim 10,wherein updating the display comprises tilting at least part of thegraphical object towards the location of the first contact or thelocation of the second contact based on the higher pressure between thefirst pressure and the second pressure.
 19. The method of claim 1,wherein the signal is based at least in part on an interaction between:the first location of the first contact and the second location of thesecond contact; and the first pressure of the first contact and thesecond pressure of the second contact.
 20. The method of claim 1,further comprising determining a gesture associated with the firstcontact and the second contact, wherein the signal is based at least inpart on the gesture.
 21. The method of claim 20, further comprisingoutputting a sound associated with the gesture.
 22. The method of claim1, wherein the touch-sensitive input device comprises the display. 23.The method of claim 1, wherein the signal is based at least in part onthe first location of the first contact corresponding to a first portionof the first object and the second location of the second locationcorresponding to a second portion of the first object.
 24. The method ofclaim 1, wherein the updating the display and the generating andoutputting the haptic signal are performed responsive to determiningthat the second sensor signal is received within a predetermined timeafter receiving the first sensor signal.
 25. The method of claim 1,wherein the signal is output based at least in part on a contactduration difference between a first contact duration of the firstcontact and a second contact duration of the second contact.
 26. Themethod of claim 1, wherein the first pressure is determined from a firstset of at least three possible pressures and the second pressure isdetermined from a second set of at least three possible pressures. 27.The method of claim 1, wherein the first contact corresponds to a firstselection of a first color and the second contact corresponds to asecond selection of a second color different than the first color, andwherein updating the display comprises displaying a mixing of the firstcolor and the second color based at least in part on the pressure of thefirst contact and the pressure of the second contact.
 28. The method ofclaim 1, wherein the touch-sensitive input device comprises a firsttouch-sensitive input device in a first electronic device and a secondtouch-sensitive input device in a second electronic device separate fromthe first electronic device, wherein the first sensor signal is receivedon the first touch-sensitive input device in the first electronic deviceand the second sensor signal is received on the second touch-sensitiveinput device in the second electronic device, and wherein the hapticeffect is output on the first electronic device.
 29. The method of claim28, wherein the haptic effect is further output on the second electronicdevice.
 30. The method of claim 1, wherein updating the displaycomprises scrolling content on the display responsive to the firstlocation of the first contact being above the second location of thesecond contact, wherein the content is scrolled upwards responsive tothe first pressure being greater than the second pressure and thecontent is scrolled downwards responsive to the second pressure beinggreater than the first pressure, and wherein the haptic effect is outputto indicate whether the content is being scrolled upwards or downwards.31. The method of claim 30, wherein a rate of scrolling of the contentis based on the pressure difference between the first pressure and thesecond pressure, and wherein the haptic effect is repeated while thecontent is scrolling.
 32. The method of claim 31, wherein a magnitude ofthe haptic effect corresponds to the rate of scrolling.
 33. The methodof claim 1, wherein the haptic effect is output in a direction from alower pressure of the first pressure or the second pressure towards ahigher pressure of the first pressure or the second pressure to indicateupdating of the display.
 34. A system, comprising: a touch-sensitiveinput device; a memory; a display corresponding to the touch-sensitiveinput device; and a processor in communication with the touch-sensitiveinput device, the memory, and the display, the processor configured to:receive a first sensor signal from the touch-sensitive input device inresponse to a first contact on the touch-sensitive input device, thefirst sensor signal comprising a first location of the first contact anda first pressure of the first contact; receive a second sensor signalfrom the touch-sensitive input device in response to a second contact onthe touch-sensitive input device substantially simultaneously with thefirst contact, the second sensor signal comprising a second location ofthe second contact and a second pressure of the second contact; updatethe display based at least in part on the first pressure being differentthan the second pressure; generate a signal configured to cause a hapticeffect indicating the updating of the display; and output the signal bywirelessly sending the signal to an arm strap comprising a haptic outputdevice configured to output the haptic effect.
 35. A non-transitorycomputer-readable medium comprising one or more software applicationsconfigured to be executed by a processor, the one or more softwareapplications configured to: receive a first sensor signal from atouch-sensitive input device in response to a first contact on thetouch-sensitive input device, the first sensor signal comprising a firstlocation of the first contact and a first pressure of the first contact;receive a second sensor signal from the touch-sensitive input device inresponse to a second contact on the touch-sensitive input devicesubstantially simultaneously with the first contact, the second sensorsignal comprising a second location of the second contact and a secondpressure of the second contact; update a display based at least in parton the first pressure being different than the second pressure; generatea signal configured to cause a haptic effect indicating the updating ofthe display; and output the signal by wirelessly sending the signal toan arm strap comprising a haptic output device configured to output thehaptic effect.